Field of the Invention
The present invention relates to a depth image correction apparatus, and more particularly to a depth image correction apparatus used for a digital camera, a digital video camera and the like.
Description of the Related Art
For such imaging apparatuses as a digital still camera and a digital video camera, imaging apparatuses having a depth measurement function, which can acquire depth from the imaging apparatus to an object at a plurality of pixel positions (hereafter this depth is called an object depth; further, an image constituted from object depths acquired at a plurality of pixel positions is called a depth image) at the same time with acquiring an ornamental image have been proposed.
One depth measurement method that is known is an imaging plane phase difference depth measurement method. According to this method, an image pickup element that can acquire at least two image signals based on an image generated by luminous flux which passed through different pupil regions of an image forming optical system is used. The object depth can be acquired by detecting the relative positional shift amount (parallax) between the two image signals by a method similar to the parallax detection method using stereo images, and converting the detected parallax into a defocus amount using a predetermined conversion coefficient. Further, according to the imaging plane phase difference depth measurement method, an ornamental image signal can be generated by combining the two image signals.
Another known depth measurement method is the depth from defocus (DFD) method. In the case of the DFD method, two image signals are consecutively acquired while changing the photographing conditions (e.g. diaphragm value, focal length), and an object depth is acquired based on the difference of blur amounts between the two images. In the DFD method, one of the two image signals can be used for the ornamental image signal.
In both of the object depth calculation methods, the object depth is calculated based on the correlation between two image signals. A region-based matching method is normally used to evaluate this correlation. In the case of the region-based matching method, an image signal included in a predetermined collation region is extracted from each image signal, and correlation thereof is evaluated. If correlation of the two images can be accurately evaluated, the object depth can be acquired very accurately, otherwise an incorrect object depth may be calculated. Therefore methods for correcting an object depth of each pixel of a depth image have been proposed.
According to Non-Patent Document 1, a depth image is corrected by a weighted cross-bilateral filter using object depth, brightness information of the ornamental image, and information indicating confidence in the object depth.
According to Patent Document 1, a region of which object depth can be calculated is classified into a plurality of sub-regions in accordance with the object depth, and a region of which object depth cannot be calculated is interpolated with object depths of a peripheral area using similarity of the ornamental image signals.    Patent Document 1: Japanese Patent No. 5066851    Non-Patent Document 1: Matsuo Takuya, Norishige Fukushima, and Yutaka Ishibashi. “Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement.” VISAPP (2). 2013.